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Fixed  and  mobile  sensors 
can  be  used  in  the  early 
detection  and  assessment 
technology  that  is  crucial 
for  homeland  security. 


The  tragic  events  of  September  11  underscored 
beyond  a  shadow  of  doubt  that  America  is  no 
longer  invulnerable  to  large-scale  terrorist  attack, 
and  the  investigations  since  conducted  into  the 
nature  of  the  threat  have  unveiled  essentially  limitless  possibil¬ 
ities  for  more  to  come.  The  potential  introduction  of  weapons 
of  mass  destruction  into  such  a  scenario  only  furthers  the  abili¬ 
ty  of  a  small  group  of  dissidents  with  relatively  limited  mili¬ 
tary  assets  to  wreak  enormous  havoc  through  asymmetrical 
warfare  or  terror.  The  principle  defense  against  surprise 
attacks  of  this  or  any  other  nature  is  advanced  warning,  which 
inherently  depends  upon  the  timely  and  accurate  collection 
and  assessment  of  appropriate  information. 

Given  the  dire  consequences  of  further 
unanticipated  or  undetected  terrorist 
action,  the  appropriate  application  of 
advanced  detection  and  assessment  technology  to  homeland 
security  is  more  than  just  critical — it  has  become  imperative. 
Business  as  usual,  no  matter  how  diligent,  will  prove  hope¬ 
lessly  inadequate  in  the  long  run.  The  threat  is  simply  too 
broad  and  diverse,  the  perpetrators  too  hard  to  isolate  and 
monitor,  and  the  vulnerabilities  they  seek  to  exploit  too 
numerous  and  geographically  dispersed.  A  system  of  robotic 
security  platforms  that  automatically  respond  in  an  adaptive 


fashion  to  potential  disturbances  reported  by  a  broad-area 
field  of  fixed  unattended  sensors  represents  a  powerful  new 
defensive  tool  for  mitigating  the  terrorist  threat. 

Background 

The  primary  purpose  of  any  robotic  system  is  to  perform  some 
useful  task  that  a  human  either  cannot  or  would  prefer  not  to 
do  and  to  hopefully  do  it  better,  cheaper,  safer,  and  more  reli¬ 
ably.  For  mobile  robots,  the  predominant  challenge  is  one  of 
perception,  in  that  the  very  nature  of  mobility  introduces  a 
never-ending  sequence  of  dynamically  changing  variables  that 
continuously  alter  the  physical  relationships  between  the  mov¬ 
ing  platform  and  its  surroundings.  Fixed- 
place  industrial  robot  applications  involving 
high-volume  repetitive  functions  (i.e.,  assem¬ 
bly,  welding,  spray  painting)  enjoy  a  definite 
advantage  in  that  the  working  environment  can  be  reasonably 
structured  to  optimize  results.  For  a  mobile  system,  such  a  pri¬ 
ori  structuring  is  simply  not  practical,  especially  in  outdoor 
environments,  due  to  the  increased  complexity  and  uncertain¬ 
ty  of  the  surroundings. 

Turning  an  autonomous  mobile  robot  loose  in  an  unstruc¬ 
tured  environment,  however,  represents  a  daunting  challenge. 
After  billions  of  dollars  invested  in  research  over  multiple 
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decades,  the  most  intelligent 
autonomous  robots  in  existence 
today  are  still  orders  of  magnitude 
less  capable  than  their  most  inept 
human  counterparts.  The  human 
body  is  an  incredible  machine,  and 
replicating  even  its  most  basic  func¬ 
tionality  is  an  arduous  task. 

Effective  perception  is  a  critical  pre¬ 
cursor  to  intelligent  behavior,  and 
fusion  of  multiple  disparate  sensor 
inputs  is  currently  the  most  effective 
way  to  emulate  human  capabilities 
in  this  regard  [1]. 

Progress  to  date  in  the  field  of  autonomous  mobile  robots 
can  be  broken  down  into  three  general  categories:  1)  move 
from  point  A  to  point  B,  2)  do  it  without  running  into  any¬ 
thing,  and  3)  perform  some  useful  mission.  Much  progress 
has  been  made  in  the  first  two  areas,  to  the  point  where 
autonomous  navigation  in  semistructured  indoor  scenarios 
has  been  realistically  achieved  [2].  Many  of  the  technical  chal¬ 
lenges  have  been  adequately  addressed  as  well  in  outdoor 
environments  and  extensively  published  in  the  literature  [3], 
[4].  Accordingly,  the  principal  focus  for  the  remainder  of  this 
article  will  be  on  category  three,  the  application  payload  that 
supports  the  mission.  For  purposes  of  illustration,  we  will 
examine  the  Mobile  Detection  Assessment  Response  System 
(MDARS)  program  managed  by  the  product  manager. 
Physical  Security  Equipment  (PM-PSE),  Ft.  Belvoir,  Virginia, 
in  that  it  represents  one  of  the  very  first  applications  to  be 
successfully  addressed  by  an  autonomous  robot. 

Early  Efforts 

The  fundamental  problem  with  automated  physical  security  sys¬ 
tems  is  as  follows:  if  the  sensitivity  is  turned  up  enough  to 
achieve  a  satisfactorily  high  probability  of  detection,  the  nuisance 
alarm  rate  also  goes  up,  and  security  personnel  subsequently 
lose  confidence  in  the  system.  The  ROBART  series  of  research 
prototypes  has  served  in  developing  the  component  technologies 
needed  to  address  this  issue  under  the  MDARS  robotic  security 
program.  While  ROBART  I  (1980-1982)  could  only  detect  a 
potential  intruder,  ROBART  n  (1982-1992)  could  both  detect  and 
assess  (Figure  1),  with  the  assessment  goal  being  the  eUmination 
of  nuisance  alarms  [5].  The  assessment  algorithm  was  supported 
by  seven  different  types  of  detection  sensors: 

1)  acoustic,  which  triangulated  a  relative  bearing  to  any 
source  of  noise 

2)  vibration,  which  picked  up  floor  vibrations 

3)  passive  infrared,  which  sensed  changes  in  thermal 
radiation 

4)  microwave,  which  detected  an  RE  Doppler  shift  due  to 
motion 

5)  optical,  which  responded  to  sudden  changes  in  ambi¬ 
ent  lighting 

6)  ultrasonic,  which  detected  changes  in  sonar  range 
indicative  of  motion 


7)  video,  which  looked  for 
motion  in  the  surveillance 
camera  field  of  view. 

The  focus  of  ongoing  research  with 
ROBART  III  (1992-)  is  more  geared 
towards  response  [6],  supported  by 
passive  infrared,  microwave,  active 
ladar,  and  video  sensors  (Figure  2). 

Each  of  the  sensor  subsystems 
listed  above  was  assigned  a  confi¬ 
dence  factor  based  on  susceptibili¬ 
ty  to  nuisance  alarms,  and  its 
effective  volumetric  coverage  was 
mapped  into  a  polar  representation  relative  to  the  robot's 
position.  ROBART  II  employed  a  four-zone  model  covering 
the  forward  180°  [5],  with  no  coverage  to  the  rear  (to  facili¬ 
tate  testing  and  demonstrations  in  a  laboratory  environ¬ 
ment).  The  assessment  algorithm  was  based  on  the 
heuristic  that  a  human  target  would  be  detected  by  multi¬ 
ple  sensors  that  showed  a  correlation  in  their  geometric 
estimation  of  disturbance  location,  whereas  spurious  nui¬ 
sance  detections  would  not  correlate.  A  composite  threat 
score  is  calculated  for  each  zone,  based  on  a  summation  of 
the  confidence-weighted  contributions  for  each  of  the 
alarmed  sensors  in  that  zone,  and  the  operator  alerted  to 
any  situation  where  the  composite  threat  exceeds  a  prede¬ 
termined  alarm  threshold. 


Fig.  1.  ROBART  II,  a  laboratory  prototype  used  to  develop  the  intelligent 
security  assessment  and  autonomous  navigation  algorithms  later  employed  on 
the  MDARS-Interior  security  robot. 
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When  the  algorithm  was  ported 
over  to  the  MDARS-Interior  robot 
(Figure  3),  the  number  of  zones  was 
increased  to  24  with  full  360°  cover¬ 
age,  and  the  optical  and  vibration 
sensors  were  eliminated  to  reduce 
complexity.  In  addition,  a  temporal 
assessment  function  was  added  to 
the  ''snapshot"  assessment  employ¬ 
ed  on  ROBART  II,  which  further 
examined  a  history  file  of  prior  zone 
activity  to  help  discriminate  pur¬ 
poseful  motion  from  random  noise. 

If  adjacent  sensors  were  found  to 
have  activated  in  a  distinct  sequence 
(i.e.,  from  left  to  right,  for  example) 
as  a  function  of  time,  indicative  of  lateral  motion  across  multiple 
zones,  then  the  composite  threat  scores  for  those  zones  were 
increased  accordingly.  Details  of  the  assessment  algorithm  and 
associated  detection  sensors  are  presented  by  Smurlo  [7].  A 
broad  agency  announcement  (BAA)  contract  was  awarded  to 
Cybermotion,  Inc.  in  1994  to  optimize  the  hardware  configura¬ 
tion  for  production,  replacing  the  bulky  staring  array  with  a 
rotating  scanner  assembly  empIo3ting  a  passive  infrared  sensor, 
Doppler  microwave,  and  flame  detector. 

State  of  the  Art 

The  most  advanced  robotic  security  system  to  date  is  seen  in 
the  MDARS-Exterior  platform  (Figure  4)  being  built  by 


Fig.  2.  ROBART  III,  the  technology  development  platform  for  pursuing 
improved  navigation  and  nonlethal  response  capabilities  for  MDARS-Interior. 


General  Dynamics  Robotic 
Systems,  Westminster,  Maryland, 
under  a  system  design  and  devel¬ 
opment  (SDD)  contract  with  PM- 
PSE.  The  initial  BAA  prototype 
employed  a  Doppler  radar  and 
infrared  (FLIR)  camera  mounted 
on  a  pan  unit  to  enable  a  step-and- 
stare  solution  to  the  need  for  360° 
coverage.  This  approach  did  not 
support  the  requirement  for  omni¬ 
directional  detection  capability,  in 
that  the  unit  was  basically  blind  to 
any  motion  outside  the  instanta¬ 
neous  (static)  field  of  view. 
Accordingly,  the  final  BAA  proto¬ 
type  employed  a  rotating  frequency-modulated  continuous- 
wave  (FMCW)  radar  developed  by  STS,  Inc.,  Scottsdale, 
Arizona.  In  this  fashion,  any  additional  targets  approaching 
from  other  directions  will  still  be  detected,  even  while  the 
ELIR  is  preoccupied  in  tracking  the  first  target. 

This  philosophy  of  high-resolution,  narrow  field-of- 
view  assessment  sensors  (in  this  case  the  ELIR)  being 
trained  on  suspected  disturbances  reported  by  wide  field- 
of-view  but  relatively  low-resolution  sensors  (i.e.,  the  360° 
scanning  radar)  is  a  key  element  of  the  MDARS  design.  The 
analogy  would  be  that  of  a  person's  focused  concentration 
on  some  area  of  interest  picked  up  by  his  or  her  peripheral 
vision  or  hearing.  (The  complementary  use  of  fixed-loca- 


Fig.  3.  The  SDD  version  of  the  MDARS-Interior  robot  patrolling  at 
Susquehannah  Army  Depot,  Pennsylvania. 
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the  nuisance  alarm  rate 
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tion  sensors  throughout  the  oper¬ 
ating  environment  to  cue  the  robot 
is  a  natural  extension  of  this  phi¬ 
losophy,  as  will  be  discussed  in 
the  next  section.) 

The  STS  millimeter-wave  radar 
does  an  excellent  job  of  detecting 
intruders  out  to  300  meters  when 
the  robot  is  not  moving,  but  its  rel¬ 
atively  narrow  vertical  field-of- 
view  limits  effectiveness  on 
nonlevel  terrain.  Accordingly,  a  tilt 
axis  was  incorporated  into  the  SDD 
version  (Figure  5)  to  automatically 
nod  the  beam  in  accordance  with  surrounding  terrain  eleva¬ 
tion  data.  To  address  intruder  detection  on  the  move,  a  par¬ 
allel  effort  is  underway  with  the  Applied  Research  Lab  at  the 
University  of  Texas  in  Austin  to  investigate  a  human-pres¬ 
ence  detector.  The  output  signals  from  passive  microwave 
and  infrared  sensors  mounted  on  a  rotating  turntable  are 
fused  together  to  isolate  a  unique  signature  based  on  human 
emissivity.  Additional  information  is  gathered  by  a  collocat¬ 
ed  laser  rangefinder  to  improve  reliability  in  this  detection 
process,  which  is  for  the  most  part  insensitive  to  the  motion 
of  the  robot.  For  improved  video  assessment,  an  interesting 
technique  under  consideration  at  the  University  of  Maryland 
seeks  to  differentiate  intruder  movement  from  that  of  a  large 
animal  by  identification  of  a  characteristic  pattern  associated 
with  the  periodic  motion  of  the  human  gait  [8]. 

Integration  with  Fixed  Sensors 

Unattended  sensors  provide  a  historically  recognized 
capability  for  effective  force  multiplication  in  security  and 
surveillance  roles,  particularly  in  the  case  of  perimeter 
defense.  Scaling  up  their  traditional  concept  of  deploy¬ 
ment  to  the  degree  necessitated  by  the  growing  terrorist 
threat,  however,  introduces  a  number  of  problems  that 
must  be  systematically  addressed  to  achieve  the  required 
success.  At  the  lowest  levels,  human  involvement  must  be 
replaced  by  intelligent  networked  sensor  arrays  that  pre- 
process  the  data,  to  eliminate  the  need  for  security  guards 
to  monitor  large  banks  of  remote  camera  displays  in  real¬ 
time,  for  example.  Otherwise  the  significant  expansion  in 
sensor  coverage  needed  to  address  effective  detection  will 
be  impractical. 

In  addition,  the  assessment  of  the  resultant  filtered  infor¬ 
mation  must  be  expanded  to  include  the  systemic  analysis 
of  the  totality  of  information,  with  a  temporal  perspective, 
to  significantly  increase  the  chances  of  pulling  out  trends  or 
patterns  that  can  focus  attention  on  suspicious  activity  that 
would  otherwise  not  be  obvious  from  a  snapshot  perspec¬ 
tive.  And  third,  the  detection  function  itself  must  be  capable 
of  immediate  adaptation  in  response  to  the  aforementioned 
assessment,  so  that  the  process  brings  more  scrutiny  to  bear 
on  any  suspicious  anomalies,  to  facilitate  even  more  accu¬ 
rate  classification. 


The  distributed  interactive 
video  array  (DIVA)  concept  pio¬ 
neered  by  the  University  of 
California,  San  Diego,  employs  a 
large-scale  redundant  cluster  of 
dispersed  video  sources  to  accom¬ 
plish  high-resolution,  broad-area 
surveillance  over  a  large  field  of 
regard  [9].  Efforts  are  now  under¬ 
way  to  integrate  DIVA  functionali¬ 
ty  into  the  MDARS  system.  From 
an  operational  perspective,  the 
fundamental  purpose  of  the  sys¬ 
tem  is  to  detect  traffic,  both  vehicu¬ 
lar  and  pedestrian  in  nature,  and  to  automatically 
investigate  any  such  detection  for  purposes  of  further  assess¬ 
ment  and  response.  The  full-volume  detection  capability  is 
supported  by  an  array  of  wide  field-of-view  sensors  with 
their  inherent  low  resolution  (i.e.,  360°  video,  wide-angle 
video,  radar,  acoustic  arrays).  High-resolution  reconfig- 
urable  sensors  such  as  zoom  cameras  or  robotic  sensors  are 


Fig.  4.  The  MDARS- Exterior  SDD  prototype  in  operation  at  the  Force 
Protection  Equipment  Demonstration  IV  (FPED  IV)  at  Marine  Corps  Air  Station, 
Quantico,  Virginia. 


Fig.  5.  Third-generation  FMCW  intrusion-detection  radar  built  by  STS,  Inc., 
for  the  MDARS-Exterior  robot. 
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Fig.  6.  Early  prototype  of  a  DIVA  sensor  employing  an  omni-directional 
camera  (top)  for  peripheral  vision  to  cue  a  high-resolution  pan-tilt-zoom 
camera  (bottom)  for  directed  surveillance  and  assessment. 


brought  to  bear  on  any  detected  anomalies  for  closer  scruti¬ 
ny  (Figure  6).  This  hierarchical  approach  retains  360°  periph¬ 
eral  awareness,  while  employing  automatic  focus-of- 
attention  with  event-driven  servoing  to  dynamically  increase 
the  effective  acuity  and  resolution  in  emergent  regions  of 
particular  interest. 

The  event-action  paradigm  allows  automatic  selection  of 
optimal  views  (perspectives  and  resolutions)  within  a  very 
large  area  of  coverage  by  defining  an  event  and  the  associat¬ 
ed  action  to  be  taken  in  response  to  that  event,  thereby 
assuring  enhanced  performance  under  varying  conditions. 
It  also  supports  dynamic  relocation  of  self-registering  sen¬ 
sors  (human  placed,  robot  placed,  or  robot  deployed)  to 
adjust  the  performance  or  focus  of  the  network  on  an  as- 
needed  basis.  For  example,  autonomous  robots  would  auto¬ 
matically  respond  to  an  area  of  suspicious  activity  to  take 
up  positions  of  optimal  perspective  to  assist  in  assessment. 
The  robot-deployed  intrusion  detection  and  assessment  sen¬ 
sor  packages  bring  enhanced  acuity  to  bear  specifically  on 
any  confirmed  sources  of  motion,  heat,  or  noise,  with  the 
added  flexibility  of  further  relocation  as  needed  to  track  a 
disturbance. 

The  concept  provides  for  synergistic  enhancement  of 
individual  sensor  performance  by  collectively  assessing  rele¬ 
vant  data  (including  that  collected  by  nonvideo-type  sen¬ 
sors)  amassed  by  the  entire  distributed  array  and  sharing  the 
appropriate  subset  of  that  information  on  an  as-needed 
basis.  The  benefits  can  be  seen  in  both  improved  assessment 
of  already  collected  information  (i.e.,  two  sensors  report  a 
similar  disturbance  with  geographical  correlation)  or  even  in 


improved  detection  probabilities  for  an  upcoming  event  (i.e., 
one  sensor  advises  another  of  an  approaching  target  in  time 
for  the  alerted  sensor  to  reconfigure  pan,  tilt,  or  zoom  to 
maximize  its  effectiveness). 

This  approach  also  avoids  needless  operator  distraction, 
which  is  critical  to  success.  Information  passed  up  the  hierar¬ 
chy  is  essentially  filtered  to  ensure  proper  response  to  an 
actual  intruder,  without  annoying  nuisance  alarms  that 
could  potentially  obscure  a  bona  fide  threat  detection.  The 
net  effect  is  that  a  human  operator  can  now  efficiently  over¬ 
see  a  significantly  larger  number  of  unattended  sensors,  and 
with  a  much  higher  probability  of  detecting  suspicious 
events  that  otherwise  could  have  been  masked  by  legitimate 
ongoing  activity  in  the  same  area. 

For  more  information  see:  www.spawar.navy- 
.mil/ robots/. 
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